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Prediction  markets  (or  information  markets)  are  an  efficient  mechanism  to  aggregate 
information  from  diverse  sources  to  develop  the  probability  of  an  event.  Such  markets  have 
accurately  forecast  presidential  elections,  Hollywood  Academy  Award  winners  and  marketing 
campaigns  in  corporations,  to  name  a  few  examples.  The  U.S.  Intelligence  Community  does  not 
lack  for  data  in  the  global  war  on  terrorism.  However,  the  9/11  Commission  Report  highlights  the 
difficulty  of  pooling  information  gathered  by  multiple  collectors  and  fusing  it  into  a  coherent, 
comprehensive  analysis  to  be  used  by  strategic  leaders.  This  paper  explores  the  viability  of 
prediction  markets  to  support  intelligence  analysis  by  coalescing  data  from  multiple  sources  into 
a  glimpse  of  the  future. 


PREDICTION  MARKETS:  ANOTHER  TOOL  IN  THE  INTELLIGENCE  KITBAG 


For  opinion  in  good  men  is  but  knowledge  in  the  making. 

— John  Milton,  Areopagitica 

The  United  States  Intelligence  Community  does  not  lack  data  relevant  to  the  global  war  on 
terrorism.  However,  the  9/11  Commission  Report 1  supports  the  findings  of  many  studies  that 
highlight  the  difficulty  of  pooling  information  gathered  by  multiple  collectors  and  fusing  it  into  a 
coherent,  comprehensive  analysis  to  be  used  by  strategic  leaders.  The  diverse  functional  areas 
and  organizational  cultures  resident  in  the  sixteen  agencies  that  make  up  our  Intelligence 
Community  underscore  this  challenge.2 


•  Central  Intelligence  Agency  (CIA) 

•  Department  of  Energy  (DOE) 

•  Department  of  Homeland  Security  (DHS) 

•  Coast  Guard  Intelligence  element 

•  Department  of  State  Bureau  of  Intelligence  and  Research  (INR) 

•  Department  of  the  T reasury 

•  Drug  Enforcement  Administration  (DEA) 

•  Federal  Bureau  of  Investigation  (FBI) 

•  Defense  Intelligence  Agency  (DIA) 

•  Army,  Navy,  Air  Force  and  Marine  Corps  Intelligence  elements 

•  National  Security  Agency  (NSA) 

•  National  Reconnaissance  Office  (NRO) 

•  National  Geospatial-Intelligence  Agency  (NGA) 

Table  1. 


The  specialized  talent  and  array  of  technology  brought  to  bear  by  each  agency  produces 
focused  analysis  and  fine-grained  comprehension  of  information.  This  distributed  approach  to 


the  production  of  intelligence  encourages  robust  compliance  with  Executive  Order  12333,  which 
directs  that 

The  United  States  intelligence  effort  shall  provide  the  President  and  the  National 
Security  Council  with  the  necessary  information  on  which  to  base  decisions 
concerning  the  conduct  and  development  of  foreign,  defense  and  economic 
policy,  and  the  protection  of  United  States  national  interests  from  foreign  security 
threats.3 

E.O.  12333  mandates  this  diversity:  “Maximum  emphasis  should  be  given  to  fostering 
analytical  competition  among  appropriate  elements  of  the  Intelligence  Community.”4  But  this 
document  poses  a  dilemma  because  it  also  requires  that  all  agencies  “cooperate  fully  to  ensure 
full  and  free  exchange  of  information  in  order  to  derive  maximum  benefit  from  the  United  States 
intelligence  effort.”5  Should  the  1C  reconcile  “competition”  with  “cooperation”? 

The  Japanese  surprise  attack  on  Pearl  Harbor  in  December  1941  and  the  employment  by 
al  Oaeda  of  suicide  aircraft  against  targets  in  New  York  and  Washington  in  September  2001 
provide  two  infamous  examples  of  the  failure  to  fuse  intelligence  collected  by  “competing” 
agencies  into  one  product  in  time  for  national  leaders  to  thwart  the  attacks.  In  October  2003,  an 
intelligence  professional  candidly  acknowledged  that  policies  and  cultural  issues  inhibit 
information  sharing  and  collaboration  between  elements  of  the  Intelligence  Community,  thereby 
posing  the  greatest  challenge  to  true  horizontal  integration  of  analysis.6  The  9/11  Commission 
Report  summed  up  the  problem  thus:  “The  biggest  impediment  to  all-source  analysis  —  to  a 
greater  likelihood  of  connecting  the  dots  —  is  the  human  or  systemic  resistance  to  sharing 
information.”7 

In  an  attempt  to  circumvent  this  selfish  human  tendency,  the  Intelligence  Community 
employs  interagency  task  forces,  working  groups  and  collaborative  centers  —  such  as  the  FBI’s 
Interagency  2004  Threat  Task  Force;  the  Technical  Support  Working  Group  which  coordinates 
research  and  development  to  defeat  terrorism;  the  Weapons  Intelligence,  Nonproliferation  and 
Arms  Control  Center;  and  the  National  Counterterrorism  Center).8  The  2001  Patriot  Act 
removed  some  of  the  legal  barriers  which  inhibited  collaboration  between  the  FBI’s  domestic 
counterterrorism  efforts  and  overseas  intelligence  collection  by  agencies  like  CIA  and  NSA.9 

But  do  we  have  a  way  to  the  next  logical  step  past  simply  sharing  data  between 
agencies?  Is  there  a  way  to  pool  the  collective  judgment  of  analysts  into  a  comprehensive 
product  for  national  security  policy-makers  and  decision-makers? 

James  Surowiecki,  a  financial  writer  for  The  New  Yorker,  makes  this  argument  in  his  2004 
book  The  Wisdom  of  Crowds:  Why  the  Many  Are  Smarter  Than  the  Few  and  How  Collective 
Wisdom  Shapes  Business,  Economies,  Societies  and  Nations : 
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What  was  missing  in  the  intelligence  community  [to  prevent  the  9/1 1  attacks], 
though,  was  any  real  means  of  aggregating  not  just  information  but  also 
judgments.  In  other  words,  there  was  no  mechanism  to  tap  into  the  collective 
wisdom  of  National  Security  Agency  nerds,  CIA  spooks,  and  FBI  agents.  There 
was  decentralization  but  not  aggregation,  and  therefore  no  organization.  ...  But 
everything  we  know  about  cognition  suggests  that  a  small  group  of  people,  no 
matter  how  intelligent,  simply  will  not  be  smarter  than  the  larger  group.  And  the 
best  tool  for  appreciating  the  collective  significance  of  the  information  that  the 
intelligence  community  had  gathered  was  the  collective  wisdom  of  the 
intelligence  community.  Centralization  is  not  the  answer.  But  aggregation  is....10 


The  Wisdom  of  the  Crowd 

H.  L.  Mencken  once  quipped,  “No  one  in  this  world,  so  far  as  I  know,  has  ever  lost  money 
by  underestimating  the  intelligence  of  the  great  masses  of  the  plain  people.”  Surowiecki  makes 
a  compelling  case  for  the  opposite  view:  large  groups  of  people  are  smarter  than  an  elite  few, 
no  matter  how  brilliant.  He  claims  that  if  four  basic  conditions  are  met,  a  crowd’s  “collective 
intelligence”  will  produce  better  outcomes  than  those  of  a  small  group  of  experts.  This  holds 
true  even  if  members  of  the  crowd  are  not  aware  of  all  the  facts  or  choose,  as  individuals,  to  act 
irrationally.  “Wise  crowds”  need  (1)  diversity  of  opinion;  (2)  independence  of  members  from  one 
another;  (3)  decentralization;  and  (4)  a  good  method  for  aggregating  opinions.11 

Diversity  brings  into  play  the  private  information  held  by  each  person,  which  may  be 
simply  their  peculiar  interpretation  of  the  known  factors.  Diversity  is  especially  important  in 
smaller  groups  because  it  adds  perspectives  that  may  not  be  inherently  present  in  a  group 
drawn  from  a  uniform  population,  such  as  an  ad-hoc  “tiger  team”  thrown  together  within  the 
same  office  to  tackle  a  problem.  A  wide  range  of  opinions  tends  to  eliminate  the  more  negative 
aspects  of  group  decision-making  (“group-think”)  and  offers  a  larger  set  of  possible  solutions  or 
novel  approaches. 

Surowiecki  maintains  that  a  group  of  highly  intelligent  people  is  not  as  effective  because 
the  members’  thought  processes  tend  to  resemble  one  another.  He  quotes  James  March,  a 
famous  organizational  theorist:  “The  development  of  knowledge  may  depend  on  maintaining  an 
influx  of  the  naive  and  the  ignorant,  and  ...  competitive  victory  does  not  reliably  go  to  the 
properly  educated.”12 

Surowiecki  cautions,  however,  that  a  group  of  diverse  but  thoroughly  uninformed  people 
will  not  produce  a  collective  decision  that  is  more  accurate  than  the  opinion  of  one  expert.  A 
diverse  group  whose  members  have  varying  degrees  of  knowledge  about  a  subject  will,  in  the 
long  run,  generate  better  solutions  than  that  single  expert.  The  expert,  like  any  human,  has  an 
ego  and  is  prone  to  overestimate  the  probability  that  he  is  correct.13 
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Independent  group  members  are  insulated  from  the  decisions  made  by  others  and  are  not 
influenced  by  a  single  opinion  leader.  This  freedom  to  think  throughout  the  crowd  provides  two 
valuable  advantages.  First,  individual  mistakes  do  not  develop  into  a  correlated  but  incorrect 
judgment.  People  become  biased  when  their  judgment  is  dependent  on  information  from  other 
parties.  Second,  information  contributed  by  independent  individuals  is  likely  to  offer  a  fresh 
perspective,  not  a  recycled  version  of  the  familiar  data.14 

Decentralization  allows  specialists  to  draw  on  local  knowledge  and  tends  to  bypass  the 
errors  inevitably  present  in  each  person’s  opinion.  Surowiecki  refers  to  this  as  tacit  knowledge 
—  that  which  can’t  be  easily  summarized  or  conveyed  to  others  because  it  is  specific  to  a 
particular  place  or  job  or  experience.15  The  U.S.  Army  champions  the  doctrine  of 
decentralization  every  day  with  its  empowerment  of  junior  leaders  to  use  their  tacit  knowledge  to 
make  decisions  that  accomplish  the  intent  of  their  commander  as  communicated  to  them 
through  his  mission-type  orders.  Successful  multinational  corporations  practice  decentralization 
when  they  empower  local  managers  in  each  country  to  shape  marketing  and  distribution 
schemes  adapted  to  the  local  culture  and  infrastructure. 

However,  decentralization  loses  its  effectiveness  if  tacit  knowledge  or  unique  intelligence 
does  not  migrate  to  the  other  people  within  a  system  who  would  benefit  from  it.  So  the  fourth 
attribute  of  a  “wise  crowd”  reinforces  decentralization:  an  aggregation  mechanism  to  mold  all  the 
private  opinions  into  a  collective  decision.  This  paper  will  explore  the  viability  for  the  U.S. 
Intelligence  Community  of  one  such  mechanism,  called  a  prediction  market  or  information 
market. 

Surowiecki  surely  warrants  such  a  consideration: 

...  [the  collective  decision’s  superiority]  rests  on  a  mathematical  truism.  If  you 
ask  a  large  enough  group  of  diverse,  independent  people  to  make  a  prediction  or 
estimate  a  probability,  and  then  average  those  estimates,  the  errors  each  of  them 
makes  in  coming  up  with  an  answer  will  cancel  themselves  out.  Each  person’s 
guess,  you  might  say,  has  two  components:  information  and  error.  Subtract  the 
error,  and  you’re  left  with  the  information.16 

The  Japanese  place  great  cultural  emphasis  on  working  as  a  group.  One  of  their  proverbs 
captures  the  spirit  of  the  “wise  crowd”  with  the  elegance  of  a  haiku:  “None  of  us  is  as  smart  as 
all  of  us.” 

Collective  Intelligence  via  the  Marketplace 

Surowiecki  is  not  the  first  observer  of  this  phenomenon.  The  Danish  philosopher  Soren 
Kierkegaard  was  commenting  on  religious  dogma  in  1846  when  he  recognized  the  tendency  for 
a  group  of  people  to  arrive  at  an  accurate  observation:  “There  is  a  view  of  life  which  holds  that 
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where  the  crowd  is,  the  truth  is  also,  that  it  is  a  need  in  truth  itself,  that  it  must  have  the  crowd 
on  its  side.”17 

A  simple  demonstration  of  collective  intelligence  was  published  in  1906  by  the  eminent 
British  anthropologist  and  statistician,  Francis  Galton.  He  was  passing  by  an  English  county  fair 
when  he  noticed  an  advertisement  for  a  contest  to  guess  the  weight  of  an  ox.  Participants  were 
invited  to  write  their  name  and  estimate  on  a  slip  of  paper,  then  the  answer  closest  to  the  actual 
dressed  weight  of  the  beast  would  win  a  prize.  The  founder  of  differential  psychology,  Galton 
studied  human  abilities  and  attempted  to  correlate  them  to  intelligence,  genetics  and  social 
factors.  He  hoped  that  the  guesses  of  these  common  farmers  might  yield  insights  into  the  ability 
of  voters  to  choose  the  best  candidate  in  a  democratic  election. 

He  was  able  to  borrow  the  800  or  so  slips  of  paper  and  discovered  that  the  mean  (or 
average)  of  all  the  estimates  fell  within  one  percent  of  the  true  weight  of  the  ox.  Galton  offered 
this  grudging  acknowledgement  that  non-intellectuals  could  construct  a  useful  collective  opinion: 
“The  result  seems  more  creditable  to  the  trustworthiness  of  a  democratic  judgment  than  might 
have  been  expected.”18 

Friedrich  Hayek,  the  British  economist  who  won  the  1974  Nobel  Prize  for  Economics, 

published  his  “Efficient-Market  Theory”  in  1948,  in  which  he  asserted  that  people  can  transmit 

their  local  knowledge  about  a  commodity  to  each  other  through  the  mechanism  of  the  market. 

The  whole  acts  as  one  market,  not  because  any  of  its  members  survey  the  whole 
field,  but  because  their  limited  individual  fields  of  vision  sufficiently  overlap  so 
that  through  many  intermediaries  the  relevant  information  is  communicated  to  all. 

The  mere  fact  that  there  is  one  price  for  any  commodity  —  or  rather  that  local 
prices  are  connected  in  a  manner  determined  by  the  cost  of  transport,  etc.  — 
brings  about  the  solution  which  (it  is  just  conceptually  possible)  might  have  been 
arrived  at  by  one  single  mind  possessing  all  the  information  which  is  in  fact 
dispersed  among  all  the  people  involved  in  the  process... The  most  significant 
fact  about  this  system  [price]  is  the  economy  of  knowledge  with  which  it  operates, 
or  how  little  the  individual  participants  need  to  know  in  order  to  be  able  to  take 
the  right  action.19 

Kenneth  Arrow  and  Gerard  Debreu  then  restricted  Hayek’s  theory  to  an  ideal  market 
which  contains  complete  information  about  the  commodities  being  traded  regardless  of  location 
or  time  period.  Their  1954  model  predicts  a  set  of  prices  such  that  aggregate  supplies  will  equal 
aggregate  demands  for  every  commodity  in  the  economy.20  Debreu  went  on  to  win  the  1983 
Nobel  Prize  for  Economics  for  his  rigorous  refinement  of  their  theory. 

In  1956,  Vernon  L.  Smith,  professor  of  economics  at  the  California  Institute  of  Technology 
(Caltech),  wanted  to  discover  if  people  with  disparate,  incomplete  knowledge  and  limited 
analytical  capability  could  marshal  resources  to  the  right  place  at  the  right  cost.  He  coached  his 
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economics  students  as  they  ran  an  imaginary  double  auction  stock  market  using  Arrow-Debreu 
securities.  In  a  double  auction,  buyers  and  sellers  shout  bids  and  asks  for  each  security,  which 
are  matched  with  shouted  responses  from  those  interested  in  accepting  them.  The  price  for 
each  mock  security  quickly  converged  to  an  optimal  price,  known  to  economists  as  the  market 
clearing  price.  Smith  was  recognized  as  a  pioneer  in  the  field  of  experimental  economics  by  the 
2002  Nobel  Prize  for  Economics. 

What  did  Smith’s  experiment  demonstrate?  Don’t  stock  and  commodity  markets  set  the 
price  of  securities  worth  billions  of  dollars  every  day,  in  much  the  same  fashion?  As  Smith 
pointed  out,  though,  these  commercial  markets  are  populated  by  professional  traders  who  are 
specialists  in  analyzing  wheat  and  pork-belly  futures,  or  airlines  and  automotive  stocks.  His 
experiment  showed  that  ordinary  people  (“naive,  unsophisticated  agents,”  as  he  referred  to 
them)  could  perform  as  well  as  the  experts  when  using  a  market  as  a  means  of  sharing 
information.  Charles  R.  Plott,  another  Caltech  economics  professor  and  Smith’s  collaborator  in 
other  work,  calls  such  a  market  an  Information  Aggregation  Mechanism ,21 

Collective  intelligence  sometimes  exhibits  itself  in  unexpected  venues.  Research  in  1984 
validated  the  long-held  notion  that  the  price  of  orange  juice  futures,  traded  by  the  Citrus 
Associates  on  the  New  York  Cotton  Exchange,  is  a  better  predictor  of  weather  in  citrus-growing 
states  than  U.S.  Weather  Service  forecasts.22  The  crash  of  the  space  shuttle  Challenger 
demonstrated  the  power  of  the  market  to  coalesce  information  into  decision.  Within  minutes  of 
the  1986  explosion,  traders  on  U.S.  stock  exchanges  dumped  shares  of  the  four  NASA 
contractors  involved  with  the  space  shuttle  program.  By  the  end  of  the  trading  day,  the  market 
had  focused  its  blame  on  the  maker  of  Challenger’s  solid-rocket  boosters:  Morton  Thiokol’s 
share  price  continued  to  plummet  while  the  other  three  non-culpable  stocks  had  regained  most 
of  their  losses.  The  averaged  opinions  of  thousands  of  traders  almost  instantaneously 
converged  on  the  culprit,  an  achievement  that  required  four  months  of  investigation  by  a 
presidential  commission.23 

What  are  Prediction  Markets? 

There  is  a  growing  body  of  literature  on  the  use  of  markets  to  merge  information  from 
disparate  sources  and  then  represent  the  result  as  a  probability.  These  markets  are  referred  to 
by  the  generic  term  prediction  markets  but  are  variably  called  information  markets,  idea  futures 
or  event  futures.  Two  business  professors  who  have  written  extensively  in  this  area,  Justin 
Wolfers  and  Eric  Zitzewitz,  describe  them  as  markets  where  “participants  trade  contracts  whose 
payoffs  are  tied  to  a  future  event,  thereby  yielding  prices  that  can  be  interpreted  as  market- 
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aggregated  forecasts.  The  price  can  be  taken  to  express  a  probability  (percentage)  of  that 
event  occurring.”24 

This  concept  is  not  without  its  critics.  Charles  Manski,  an  economics  professor  at 
Northwestern  University,  asked  in  2004,  “What  is  the  logical  basis  for  interpreting  the  price  of  an 
all-or-nothing  futures  contract  as  a  market  probability  that  the  event  will  occur?”25  But  his 
contention  was  specifically  rebutted  by  Wolfers  and  Zitzewitz  in  2006.  Using  a  formal  model  , 
they  demonstrated  that  prediction  market  prices  are  usually  close  to  the  mean  estimates  of 
traders. 26 

Robin  Hanson,  a  George  Mason  University  economics  professor,  established  Foresight 
Exchange  in  1994,  a  seminal  prediction  market  still  in  operation  today.27  He  is  credited  with 
coining  the  term  ideas  futures  in  1999.  He  advocated  trading  contingent  contracts  linked  to  the 
outcomes  of  more  than  one  event  in  decision  markets  to  examine  the  correlation  between 
events.  Hanson  concluded  that  such  derivative  contracts  can  estimate  the  net  effect  of  any 
policy  choice  on  any  outcome  of  interest  that  can  be  verified  after  the  fact.28 

The  Department  of  Defense  is  no  stranger  to  prediction  markets.  In  1968,  a  team  led  by 
Dr.  John  Craven,  Chief  Scientist  of  the  U.S.  Navy's  Special  Projects  Division,  was  tasked  to 
search  for  the  nuclear  submarine  USS  Scorpion,  then  missing  in  the  Atlantic  Ocean.  He 
organized  an  internal  prediction  market  among  the  scientists,  salvage  experts  and  submarine 
officers  on  his  team  to  solicit  their  best  guesses,  given  the  scenario  surrounding  the  sub’s 
sinking.  Craven  employed  Bayes’  theorem  —  useful  for  calculating  the  impact  of  new 
information  on  a  prediction  —  to  aggregate  their  guesses  into  a  starting  point  for  the  search. 
After  a  five-month  effort,  Scorpion’s  final  resting  place  was  discovered  to  be  220  yards  from  the 
group’s  starting  point.29 

DoD’s  First  Intelligence  Prediction  Market:  FutureMAP 

Let’s  return  to  our  question  of  intelligence  aggregation  for  policy-makers.  A  search  for 
such  a  system  had  been  underway  within  the  Department  of  Defense  for  five  months  when  the 
al  Qaeda  attacks  occurred  in  September  2001.  The  Defense  Advanced  Research  Projects 
Agency  (DARPA)  issued  a  call  for  proposals  in  May  2001  under  the  title  of  “Electronic  Market- 
Based  Decision  Support,”  later  renamed  “Future  Markets  Applied  to  Prediction  (FutureMAP)”. 
The  program  objective  stated  that  FutureMAP  would  concentrate  on  market-based  techniques 
for  avoiding  surprise  and  predicting  future  events.  It  recognized  that  independent  contributions 
from  experts  in  a  wide  variety  of  fields  would  have  to  be  combined  into  one  assessment  on 
topics  such  as  political  stability  in  regions  of  the  world  and  the  impact  on  national  security  of 
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emerging  technologies.  DARPA  also  noted  that  the  rapid  reaction  of  markets  to  knowledge  held 
by  only  a  few  participants  may  provide  an  early  warning  system  to  avoid  surprise.30 

A  company  called  Net  Exchange  (with  Robin  Hanson  as  subcontractor)  won  a  contract 
with  its  proposal  for  a  Policy  Analysis  Market  (PAM)  to  focus  on  the  key  Middle  East  nations  of 
Turkey,  Egypt,  Saudi  Arabia,  Israel,  Syria,  Jordan,  Iraq  and  Iran.  This  Internet-based  PAM 
would  establish  trading  markets  to  cover  the  economic  health,  civil  stability  and  military  posture 
of  each  country,  along  with  their  economic  interactions  and  military  involvements  with  the  United 
States.  Each  of  these  markets  would  track  a  financial  index  customized  by  the  Economist 
Intelligence  Unit  (EIU),  a  subsidiary  of  The  Economist  magazine.31 

At  the  start  of  each  quarter,  the  Policy  Analysis  Market  would  issue  securities  which  stated 
that  the  EIU  index  in  each  market  for  each  country  would  exceed  a  published  target.  Traders 
would  buy  and  sell  contracts  based  on  their  knowledge  about  this  result.  Constrained  between 
$0  and  $100,  the  final  price  of  the  contract  would  be  analogous  to  the  probability  (expressed  as 
a  percentage)  of  that  event  occurring.  Each  security  would  mature  in  one  year,  whereupon  the 
EIU  would  adjudicate  the  results.  The  PAM  would  then  pay  off  the  winning  predictions.32 

Net  Exchange  proposed  two  other  types  of  futures  contracts  to  be  offered  under  the  Policy 
Analysis  Market.  One  type  would  track  global  economic  and  conflict  indicators,  such  as  the 
U.S.  stock  market,  the  U.S.  Gross  Domestic  Product,  global  trade,  deaths  attributed  to 
terrorism,  and  U.S.  military  casualties.  A  second  category  would  offer  contracts  on  specific 
possible  events,  such  as  American  recognition  of  a  Palestinian  State  in  the  first  quarter  of  2005. 
Traders  could  also  construct  derivative  securities  that  combined  the  different  markets:  “If 
Turkey’s  ‘military  involvement  with  the  U.S.’  index  tops  X,  will  Syria’s  ‘military  posture’  index 
exceed  Y?”33 

Once  traders  expressed  interest  in  a  specific  event,  Net  Exchange  would  nominate  a 
definition  to  the  EIU,  stated  as  “yes/no”  or  “happened/did  not  happen.”  If  the  EIU  accepted  the 
responsibility  of  assessing  the  security  at  maturity,  Net  Exchange  would  place  the  new  security 
into  trade  on  the  Policy  Analysis  Market  website. 

The  following  table,  provided  during  a  briefing  by  Net  Exchange  in  June  2003,  illustrates 
how  trading  in  these  securities  might  work. 34 
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Security  listed  on 

15  Dec  2003 

Posted 

Prediction 

Trader  and  Perception 

Trader  Action 

Turkish  econ.  health  in 
Q2  ’04  will  be  >  1 15 
(an  index  measure) 

360 

Bank  officer  charged  with  Turkey 
risk  assessment  considers 
this  more  likely 

Buy  contracts  in 
this  security  until 
prediction  =  450 

Civilian  deaths  from 
terrorism  in  Q4  ‘04 

500 

Reporter  with  a  Middle 

Eastern  news  agency 
considers  this  too  high 

Sell  contracts  until 
prediction  =  300 

The  U.S.  recognizes  a 
provisional  Palestinian 
state  in  Q3  ‘04 

550 

Professor  of  Middle  Eastern 
studies  thinks  this  is  high 

Sell  contracts  until 
prediction  =  400 

If  Q3  ’04  Iraqi  econ. 
health  >  150, 
then  Q4  ’04  Syrian  civil 
stability  <  100 

420 

Expert  in  Syrian 
domestic  politics 
considers  this  far  too  low 

Buy  contracts  until 
prediction  =  600 

Table  2. 


Net  Exchange  intended  to  operate  the  PAM  under  two  different  regimes.  PAM  #1  would 
be  restricted  to  about  2000  traders  who  were  employees  of  U.S.  Government  agencies.  PAM 
#2  would  expand  their  number  to  include  an  additional  6000  traders,  who  would  add  the 
diversity  of  tacit  knowledge  found  in  academia,  the  media,  policy  research  institutes  (“think 
tanks”),  insurance  companies,  non-governmental  organizations  and  the  general  public.  Net 
Exchange  hoped  to  recruit  half  of  the  traders  in  PAM  #2  from  outside  the  United  States  — 
eventually  including  officials  of  foreign  governments.35 

FutureMAP’s  debut  in  July  2003  met  with  harsh  criticism  from  Congress  and  the  media. 
Their  tirades  centered  on  the  misconception  that  the  PAM  would  be  a  forum  for  traders  to  profit 
from  assassinations  and  terrorist  attacks,  a  ghoulish  waste  of  public  money  which  would 
produce  no  public  benefit.  DARPA  canceled  the  project  within  a  few  days  but  the  media 
posturing  endured  for  several  months.  At  the  time  of  FutureMAP’s  demise,  Net  Exchange  was 
recruiting  the  first  cohort  to  begin  trading  using  their  company’s  proprietary  market-making 
software;  the  project  had  expended  a  total  of  about  $1  million.36 

Opposition  to  the  “Terrorism  Futures  Market” 

The  arguments  against  the  Policy  Analysis  Market  fell  into  seven  general  categories, 
according  to  Robin  Hanson37  and  Robert  Looney.38  It  is  useful  to  examine  each  one  before 
deciding  the  utility  of  prediction  markets  for  this  task. 

(1 )  The  system  creates  a  strong  incentive  for  someone  to  buy  futures  in  a  violent  act  and 
then  carry  out  the  act:  the  Insider  Information  Problem.  This  possibility  is  not  new;  it  is  common 
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knowledge  that  al  Qaeda  is  suspected  of  selling  short  their  stocks  of  airline  and  hotel  companies 
just  prior  to  their  September  2001  airplane  attacks.  However,  the  small  dollar  amounts  of  trades 
to  be  conducted  in  the  Policy  Analysis  Market  made  it  doubtful  that  terrorists  could  realize  any 
significant  funding  by  betting  on  their  own  operations.  Terrorists  trading  in  the  PAM  would 
actually  make  the  information  content  of  the  price  more  realistic. 

(2)  The  system  would  create  incentives  for  participants  to  manipulate  the  market  by 
spreading  rumors  about  the  likelihood  of  events.  This  argument  concerns  an  event  which  would 
not  be  the  main  focus  of  the  Policy  Analysis  Market,  such  as  the  assassination  of  a  foreign  head 
of  state.  The  market  automatically  takes  into  account  that  the  head  of  state’s  government  would 
take  action  to  pre-empt  this  act,  so  there  would  be  no  dramatic  increase  in  the  share  price  from 
which  a  terrorist  could  quickly  profit.  As  noted  previously,  most  PAM  contracts  would  have 
covered  an  index  linked  to  a  long-term  indicator,  not  a  specific  event  that  could  be  easily  altered 
or  manipulated. 

(3)  The  PAM  would  suffer  from  insufficient  trader  participation.  Joseph  Stiglitz,  winner  of 
the  2001  Nobel  Prize  for  Economics,  argued  that  the  PAM  would  lack  fidelity  because  it  would 
be  a  “thin”  market.39  He  referred  to  his  1976  study  conducted  with  Sanford  Grossman40  in  which 
they  contended  that  the  Efficient-Market  Theory  was  idealistic.  They  claimed  that  because 
acquiring  information  to  make  knowledgeable  trades  is  costly,  prices  cannot  perfectly  reflect  all 
available  information.  If  prices  were  truly  Hayekian,  then  traders  who  spent  resources  to  obtain 
information  would  receive  no  compensation.  Grossman’s  and  Stiglitz’  hypothesis  would  act  as 
an  economic  disincentive  to  potential  traders  in  the  PAM.  They  also  challenged  the  assumption 
that  the  market  price  would  accurately  capture  the  probability  of  the  event  covered  by  the 
contract.  However,  Jed  Christiansen’s  2006  research,  using  prediction  markets  which  covered 
five  British  and  international  rowing  regattas,  demonstrated  remarkable  market  calibration  with 
participation  by  as  few  as  sixteen  traders.41 

(4)  Participants  would  have  doubts  that  anyone  could  ever  cash  in  on  a  future  contract,  as 
it  is  likely  that  public  outrage  would  not  allow  it.  Imagine  Congress  allowing  a  trader  to  collect 
his  winnings  after  successfully  predicting  the  September  2001  al  Qaeda  attacks!  Again,  this  is  a 
fallacious  argument  concerning  a  contract  not  likely  to  be  offered  on  the  Policy  Analysis  Market. 
If  DoD  ever  refused  to  pay  off  a  contract,  no  one  would  again  participate  in  the  PAM;  such  a 
refusal  would  defeat  the  purpose  of  the  market,  which  is  to  solicit  intelligence. 

(5)  Futures  contracts  can  be  written  only  for  events  that  are  explicitly  anticipated.  Given 
the  inexhaustible  creativity  of  the  human  mind,  many  events  of  interest  to  policy-makers  would 
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be  difficult  to  anticipate.  But,  again,  most  of  the  PAM’s  securities  would  cover  leading  indicators 
of  the  strategic  environment  rather  than  spectacular  incidents. 

(6)  The  PAM  is  a  gambling  parlor.  Net  Exchange  negotiated  with  the  Securities  and 
Exchange  Commission,  the  Commodity  Futures  Trading  Commission  and  the  Internal  Revenue 
Service  to  limit  transactions  to  $100,  thus  staying  under  the  threshold  of  U.S.  gambling  laws  and 
futures-trading  regulations.  Furthermore,  the  gambling  laws  do  not  apply  to  speculators  in 
futures  markets  who  often  risk  sums  of  millions  of  dollars.  The  difference  between  the  definition 
of  gambling  and  speculating  is  the  degree  of  chance  regarding  the  outcome  —  speculators  risk 
money  when  they  have  a  high  level  of  confidence  that  they  will  be  rewarded. 

(7)  The  PAM  would  be  inequitable.  Another  accusation  put  forward  by  Stiglitz  portrayed 
the  Policy  Analysis  Market  as  a  ploy  by  the  Bush  Administration  to  allow  rich  Americans  to 
hedge  against  terrorism  events  by  playing  the  PAM,  while  the  majority  of  U.S.  citizens  would 
remain  ignorant  and  financially  vulnerable.42  But  the  PAM  limit  on  the  size  of  bets  prevents  the 
wealthy  from  profiting  in  much  the  same  way  as  it  restricts  the  terrorists  from  realizing  any 
substantial  monetary  gains.  Further,  rich  people  use  insurance  to  protect  themselves  against 
economic  loss. 

Prediction  Markets  in  the  Business  World 

Numerous  examples  of  successful  prediction  markets  in  the  commercial  and  academic 
world  prompted  DARPA  to  explore  this  option  for  intelligence  collection.  Puong  Fei  Yeh,  writing 
in  the  December  2006  issue  of  the  CIA’s  Studies  in  Intelligence,  provides  a  comprehensive 
survey  of  prediction  markets  in  general  and  the  PAM  in  particular.43  Consider  the  following 
examples: 

The  Iowa  Electronic  Markets  (I EM)44  have  been  operated  as  an  educational  and 
research  project  since  1988  by  the  Henry  B.  Tippie  College  of  Business  at  the  University  of 
Iowa.  Elections  contracts  in  the  IEM  cover  U.S.  Presidential  and  Congressional  elections, 
selected  U.S.  state  and  foreign  elections,  and  economic  matters  ranging  from  corporate 
earnings  and  the  Mexican  peso  to  the  monetary  policy  of  the  Federal  Reserve  Open  Market 
Committee  (“The  Fed”).  Trade  prices  are  constrained  between  $0.00  and  $1.00  in  increments  of 
$0,001  to  reflect  a  probability  between  zero  and  100%.  The  IEM  capped  accounts  at  $500  in 
order  to  qualify  for  a  letter  of  “no  regulatory  action”  from  the  Commodity  Futures  Trading 
Commission.45  The  IEM  economic  markets  are  specifically  restricted  to  traders  from  academia, 
but  the  general  public  may  participate  in  any  of  the  elections  markets.  The  latter  exhibit 
impressive  correlation  between  IEM  predictions  and  actual  outcomes  for  elections,  with  errors 
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ranging  between  1.37%  and  3.44%.  The  I  EM  elections  markets  have  performed  about  as  well 
as,  or  slightly  better  than,  the  average  of  several  national  elections  polls.46 

Started  in  1996,  the  Hollywood  Stock  Exchange  (HSX)  calls  itself  “the  world’s  longest 
continuously  operating  commercial  prediction  market.”  Tens  of  thousands  of  film  aficionados 
around  the  world  regularly  buy  and  sell  contracts  using  virtual  “Hollywood  dollars”  (no  monetary 
value)  to  back  their  predictions  of  Oscar  and  Grammy  winners,  movie  star  popularity,  and  box- 
office  receipts  for  movies.  Now  a  unit  of  the  financial  services  company  Cantor  Fitzgerald,  HSX 
sells  its  predictions  of  long-term  and  short-term  film  revenues  to  studios  and  producers.47 

NewsFutures  advertises  itself  as  the  leading  provider  of  prediction  markets  worldwide 
(50,000  markets  since  2000)  under  the  slogan  “Trading  uncertainty  for  collective  wisdom.”  It 
runs  no-cost  predictions  markets  to  enable  the  public  to  bet  on  such  topics  as: 

•  Palestine  is  recognized  by  the  UN  as  a  member  state  on  or  prior  to  January  1 ,  2008 

•  Nouri  al-Maliki  will  remain  Prime  Minister  of  Iraq  through  the  rest  of  Bush's  presidency 

•  The  U.S.  will  catch  Osama  bin  Laden  while  Bush  is  president 

•  Islamic  terrorists  will  attack  in  the  U.S.  before  attacking  within  the  European  Union 

•  Islamic  terrorists  will  hit  the  U.S.  homeland  again  while  Bush  is  president.48 

NewsFutures  also  designs  and  operates  prediction  markets  for  corporate  clients  to  help 

them  gain  insight  into  sales  and  strategic  commodity  price  forecasting,  new  product 
development,  dates  of  project  completions,  business  intelligence,  and  industry  trend-spotting. 
Their  clients  include  industry  leaders  such  as  Yahoo!,  SAIC,  Siemens,  Lilly,  Corning,  Dentsu 
(Japan’s  largest  advertising  firm),  and  Arcelor  (a  European  and  South  American  steel 
producer)49  Finally,  NewsFutures  operates  the  virtual-money  Global  Risks  Prediction  Market 
on  behalf  of  the  World  Economic  Forum  in  Geneva,  Switzerland.50 

Inkling  Markets,  founded  in  Chicago  in  2005,  advertises  itself  as  an  online  service  for 
individuals  and  organizations  to  create  and  manage  their  own  prediction  markets.  Its  services 
have  been  used  by  both  corporations  and  academic  researchers.51 

Intrade  bills  itself  as  “the  leading  public  prediction  market  platform,  with  over  65,000 
eligible  members  from  over  125  countries  generating  predictions  on  thousands  of  event  futures 
since  2000.”  Intrade,  incorporated  in  Ireland,  handles  for-profit  bets  on  politics,  economics, 
current  events  and  the  weather,  requiring  an  initial  deposit  of  $250.  It  is  a  subsidiary  of  the  Irish 
on-line  betting  firm  Tradesports;  both  entities  are  exempt  from  U.S.  gambling  and  securities 
laws.52 

Hedge  Street  calls  itself  “the  first  Internet-based,  government  regulated  market  where 
traders  can  hedge  against  or  speculate  on  economic  events  and  price  movements.”  The 
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company  is  registered  as  a  futures  exchange  with  the  Commodity  Futures  Trading  Commission 
and  allows  consumers  to  trade  in  small  futures  contracts  ranging  from  commodities  and 
currencies  to  economic  indicators  (such  as  employment,  fuel,  housing  prices,  inflation,  and 
interest  and  mortgage  rates).  One  novel  option  allows  workers  to  try  to  offset  their  commuting 
costs  by  hedging  with  weekly  futures  contracts  predicting  gasoline  prices.53 

At  this  point,  a  discerning  reader  will  ask:  Does  the  accuracy  of  a  prediction  market 
depend  on  the  use  of  real  money  versus  virtual  trading  currency?  Servan-Schreiber  et  al. 
compared  the  predictions  produced  on  Tradesports  (actual  money)  with  NewsFutures  (play 
money)  during  the  2003  season  of  the  National  Football  League;  both  markets  displayed  equal 
accuracy.54  Many  traders  seem  to  be  motivated  simply  by  the  prestige  of  success. 

The  growing  academic  and  commercial  interest  in  prediction  markets  is  reflected  in  the 
Prediction  Markets  Cluster,  whose  mission  statement  calls  it  “the  open  industry  commons  and 
consortium  for  prediction  markets  ...  a  collaboration  of  vendors,  academia,  traders,  users, 
developers,  markets,  regulators  and  stakeholders.”  Besides  corporate  and  academic  sponsors, 
it  lists  recent  conferences  covering  prediction  markets:55 

•  Prediction  Markets  Conference,  18  January  2007,  sponsored  by  the  American 
Enterprise  Institute-Brookings  Joint  Center,  Washington,  DC;  proceedings  available  at 
http://www.pmcluster.com/IAD.htm. 

•  Prediction  Markets:  Tapping  the  Wisdom  of  Crowds,  13  December  2006,  sponsored 
by  Yahoo!  Technology  Development  Group,  Sunnyvale,  California;  proceedings 
available  at  http://confab.yahoo.com. 

•  Information  and  Prediction  Markets  —  European  Summit,  2  November  2006, 
sponsored  by  Colabria,  Vienna,  Austria;  proceedings  available  at  http://pmcluster.com/ 
VIE.htm. 

The  American  Enterprise  Institute-Brookings  Joint  Center  in  Washington,  DC 
maintains  a  webpage  featuring  its  own  research  on  information  markets,  notably  a  book  co¬ 
authored  by  several  contemporary  scholars  referenced  in  this  paper. 56  This  webpage  also 
highlights  other  scholarly  articles,  op-ed  pieces,  and  a  comprehensive  listing  of  other  websites 
concerning  prediction  markets. 57 

Recommendations  for  Implementing  Prediction  Markets  in  the  Intelligence  Community 

(1)  Designate  several  analysts  from  each  of  the  16  members  of  the  national  Intelligence 
Community  to  trade  on  contracts  of  interest  in  existing  commercial  prediction  markets 
(NewsFutures,  Intrade,  Inkling).  For  markets  requiring  real  money,  each  agency  could  bankroll 
its  traders  with  mission  funds  and  allow  them  to  keep  any  winnings  as  a  performance  award. 
Track  the  results  of  these  contracts. 
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(2)  Create  a  prediction  market  as  a  classified  intelligence  aggregation  mechanism,  similar 
in  structure  to  PAM  #1 .  It  would  be  open  only  to  employees  of  the  Intelligence  Community  and 
selected  outside  experts  with  the  appropriate  security  clearance.  This  prediction  market  would 
cover  the  same  type  of  contracts  described  earlier  in  this  paper  for  the  original  PAM.  The 
Intelligence  Community  should  find  it  useful  to  calibrate  this  market’s  results  with  appropriate 
analogous  contracts  traded  in  the  market  of  Recommendation  #1 . 

One  legal  challenge  is  gaining  authorization  for  the  transfer  of  appropriated  funds  between 
government  agencies  without  the  approval  of  Congress.  Under  PAM,  Net  Exchange  proposed 
that  DoD  act  as  the  market-maker  and  provide  all  traders  (from  all  agencies)  with  funds  from  a 
DoD  appropriated  budget  line.  Each  trader  would  be  able  to  keep  any  profits;  winnings  of  DoD 
employees  would  be  booked  as  performance  awards  to  account  for  the  internal  transfer  of  funds 
from  the  DoD  mission  account  to  its  personnel  account.58 

(3)  Revive  PAM  #2  under  a  new  name.  Devise  a  carefully-planned  strategic 
communications  campaign  to  forestall  the  unfavorable  publicity  of  its  first  incarnation.  Recruit 
participants  from  a  diverse  cross-section  of  society,  using  the  same  criteria  as  PAM  #2  to  select 
about  8000  traders.  To  provide  comparison,  structure  the  contracts  to  mirror  appropriate  issues 
tracked  in  the  classified,  U.S.  Government-only  market  proposed  in  Recommendation  #2. 

(4)  Senior  officials  at  each  intelligence  agency  must  support  participation  of  their  traders 
and  be  willing  to  incorporate  the  predictions  produced  by  the  various  markets  into  the 
intelligence  fusion  process. 

Conclusion 

Prediction  markets  a  la  the  Policy  Analysis  Market  will  not  reveal  brilliant  insights  about 
terrorist  intentions;  they  won’t  provide  tactical  initiative  but  may  prevent  strategic  surprise.  The 
final,  probabilistic  prices  of  the  various  contracts  can  act  as  a  signal  for  strategic  leaders  about 
where  intelligence  efforts  should  be  more  focused. 

The  professed  Congressional  unease  that  the  original  PAM  would  allow  people  to  profit  by 
a  market  in  death  and  destruction  is  myopic  and  nonsensical.  Such  a  market  has  existed  for 
hundreds  of  years  —  it's  called  the  insurance  industry.  Every  day,  insurance  companies  earn 
money  by  predicting  who  will  die  and  when,  and  what  kind  of  disaster  might  strike. 

A  similar  logic  can  be  applied  to  the  concern  that  terrorists  might  profit  in  an  intelligence 
prediction  market  by  using  their  tacit  knowledge.  Paying  money  to  unsavory  characters  in 
exchange  for  intelligence  is  nothing  new;  the  Sanhedrin  paid  Judas  Iscariot  thirty  pieces  of  silver 
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for  providing  the  time  and  place  that  Jesus  would  be  alone  in  Gethsemane.59  An  intelligence 
prediction  market  is  just  another  economic  enticement  to  elicit  information  from  human  sources. 

Terrorist  movements  exhibit  enormous  creativity  in  inventing  new  ways  to  attack  the 
United  States.  Let’s  give  our  Intelligence  Community  every  possible  tool  to  defeat  them. 
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